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WATERMARKED IMAGE 
GENERATOR AND METHOD OF 
EMBEDDING WATERMARKS 
INTO AN INPUT IMAGE 

Background of the Invention 
Field of Invention 

[0001] This invention is related to embedding watermarks in input images to form 
watermarked output images. 

Description of Related Art 

[0002] With the advent of digitization of images, digital image distribution and digital 
video availability, the "hiding" of information in digital images for purposes such as 
copyright protection has become a substantial issue for image publishers and authors. 
The process of imbedding information in a digital image is known as "watermarking". 
Such watermarks must be secure, robust to intentional corruption and to compression 
processing, not unreasonably complex to embed and extract, and compatible and 
interoperable with conventional image processing systems. The watermark is 
generally invisible to a viewer. However, in some applications, it is desirable to 
produce a visible watermark that can be removed by an authorized image decoder and 
that can not be removed by an unauthorized decoder. 

[0003] Many watermark algorithms have been proposed. However, only a few watermark 
algorithms have been designed for printed images. U.S. Patent 5,734,752 to Knox 
discloses a method of using stochastic screens to create correlation-based digital 
watermarks. The concept of conjugate halftone screens for watermarking is proposed 
in U.S. Patent 5,790,703 to Wang. The methods disclosed in Knox and Wang might not 
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be suitable for applications which require dynamically changing watermark 
information, because the watermarks in those methods can only be embedded during 
the screen design process for the halftone screen that will be used when printing a 
given image. U.S. Patent 5,946,1 03 to Curry discloses a method that uses glyphs to 
digitally watermark a printed document. However, glyph encoding typically generates 
images with noticeable structures. This makes this method suitable only for specific 
applications. 

Summary of the Invention 

[0004] This invention provides systems and methods that embed watermarks in input 
images to produce output images having relatively high watermark resolution. 

[0005] This invention separately provides systems and methods that embed watermarks 
in input images to produce output images having a relatively small amount of 
noticeable artifacts. 

[0006] This invention separately provides systems and methods that dynamically embed 
any type of watermark in an input image at run-time. 

[0007] In various exemplary embodiments of the systems and methods according to this 
invention, a watermark image generator includes a watermark data source that inputs 
watermark data to a watermark embedding device. The watermark embedding device 
halftones the input image to generate the output image made of 2x2 binary patterns. 
The 2x2 binary patterns form the watermarks embedded in the output image. The 
watermark embedding device includes a tri-level error diffusion generator that 
performs tri-level error diffusion on the input image by halftoning the input image 
into black, white and 50% gray. A plurality of halftoning circuits successively replace 
each pixel of the halftoned input image with one of a plurality of the 2x2 binary 
patterns. The one of the plurality of 2x2 binary patterns corresponds to at least one 
respective bit of the watermark data when a pixel of the halftoned image data is 50% 
gray. 

[0008] in various exemplary embodiments of the watermarked image generator according 
to this invention, when a pixel of the halftoned input image data is 50% gray, the one 
of the 2x2 binary patterns forms one of a plurality of polarized patterns. The one of 
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the plurality of polarized patterns is polarized according to the value of the respective 
bit of the watermark data. 

[0009] These and other features and advantages of this invention are described in, or are 
apparent from, the following detailed description of various exemplary embodiments 
of the systems and methods according to this invention. 

Brief Description of the Drawings 

[0010] Various exemplary embodiments of this invention will be described in detail, with 
reference to the following figures, wherein: 

[0011] Fig. 1 shows a first exemplary embodiment of the watermarked image generator 
according to this invention; 

[001 2] Fig. 2 shows a first exemplary embodiment of a watermark embedding device 
according to this invention; 

[001 3] Fig. 3 shows the 2x2 binary patterns generated by the watermarked image 
generator according the first exemplary embodiment of this invention; 

[001 4] Fig. 4 is a flowchart illustrating a first exemplary embodiment of the method of 
embedding watermarks in an input image according to this invention; 

[001 5] Figs. 5A-5D show patches of output images after being watermarked by the first 
exemplary embodiment of the watermarked image generator according to this 
invention; 

[001 6] Fig. 6 is an error diffused image of a chapel incorporating a digital watermark 
generated according to this invention; 

[001 7] Fig. 7 A shows a portion of the digital version of the image shown in Fig. 6 after 
being watermarked by the first exemplary embodiment of the method of embedding 
watermarks in an input image according to this invention; 

[001 8] Fig. 7B shows the same portion of the image shown in Fig. 6 that is shown in Fig. 
7a after being printed and scanned; 

[001 9] gA s h ows a p 0rt i on Q f the digital version of the image shown in Fig. 6 after 
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being watermarked by the first exemplary embodiment of the method of embedding 
watermarks in an input image according to this invention; 

[0020] Fig. 8B shows the same portion of the image shown in Fig. 6 that is shown in Fig. 
8A after being printed and scanned; 

[0021] Fig. 9 shows one region of the image shown in Fig. 6 after being embedded with a 
bar code by the first exemplary embodiment of the method of embedding watermarks 
in an input image according to this invention; 

[0022] Fig. 1 0 shows another region of the image shown in Fig. 6 after being embedded 
with a bar code by the first exemplary embodiment of the method of embedding 
watermarks in an input image according to this invention; 

[0023] Fig. 1 1 shows the image shown in Fig. 6 after being embedded with a bar code by 
the first exemplary embodiment of the method of embedding watermarks in an input 
image according to this invention; 

[0024] Fig. 1 2 shows a second exemplary embodiment of the watermark embedding 
device according to this invention; 

[0025] Fig. 1 3 shows the 2x2 binary patterns generated by the watermarked image 
generator according to the second exemplary embodiment of this invention; 

[0026] Fig. 1 4 is a flowchart illustrating a second exemplary embodiment of the method 
of embedding watermarks in an input image according to this invention; 

[0027] Fig. 1 5 shows a segmentation map embedding device that encodes a 

segmentation map before embedding the segmentation map into an image; 

[0028] Fig. 1 6 is a flowchart illustrating an exemplary embodiment of the method of 
embedding watermarks in an input image according to this invention; 

[0029] Fig. 1 7 shows a printed image of an advertisement; 

[0030] Fig. 1 8 shows a segmentation map of the image shown in Fig. 1 7; 

[0031] Fig. 1 9 shows the image of Fig. 1 7 after being embedded with its segmentation 
map by an exemplary embodiment of the method of embedding watermarks in an 
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input image according to this invention; and 

[0032] Fig. 20 is a flow chart illustrating the embedding of binary segmentation values 
into an input image using the second embodiment of the method for embedding a 
watermark into an image according to this invention. 

Detailed Description of Preferred Embodiments 

[0033] The various exemplary embodiments of the methods and apparatus according to 
this invention use a watermarking algorithm known as bipolar encoding through error 
diffusion. This algorithm employs polarized dot structures for data encoding. 

[0034] Fig. 1 shows one exemplary embodiment of a watermarked image generator 1 00 
for embedding watermarks in an input image according to this invention. As shown in 
Fig. 1 , an input image is input from an image data source 200 over a link 21 0 to the 
watermarked image generator 100. It should be appreciated that the image data 
source 200 can be any device that stores and/or generates an electronic version of an 
image. 

[0035] Thus, the image can be a printed hard copy version of the image, and the image 
data source 200 can be a scanner that scans and outputs an electronic version of the 
image over the link 21 0 to the watermarked image generator 1 00. Furthermore, the 
image data source 200 and the watermarked image generator 1 00 can be elements 
integrated into a photocopier. 

[0036] Similarly, the image data source 200 can be a server or other node on a local area 
network, a wide area network, an intranet, the Internet or any other distributed 
network. In this case, the image is already stored on the network in electronic form. 

[0037] Finally, the link 21 0 can be a wired or wireless link to the image input device 200. 
Thus, the image data source 200 and the link 210 can be any known or later- 
developed elements that are capable of supplying an electronic image to the 
watermarked image generator 100. 

[0038] As ^cussed above, the system and methods of this invention work equally well 
on images that have not been transferred to hard copy. In this case, the image is 
already in digital form and the image is ready for processing by the watermarked 
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[0039] The watermarked image generator 1 00 includes an input interface 11 0, a 

controller 1 20, a memory 1 30, one or more user input devices 1 40, and a watermark 
embedding device 1 50. The one or more user input devices 140, the image input 
device 200 and an image data sink 300 are connected to the watermark embedding 
device 100 via the input/output interface 1 10. 

[0040] In operation, grayscale image data is received from the image input device 200 via 
the link 210 and the input/output interface 110 of the watermarked image generator 
100. The input/output interface 1 10 forwards the input image data received from the 
image input device 200, under the direction of the controller 1 20, to the watermark 
embedding device 1 50. A watermark is stored in the memory 1 30, or a particular 
watermark can be input by the user through the user input device 140 and/or the 
image data source 200. The watermark embedding device 1 50 halftones the input 
image data to output an image made up of 2x2 binary patterns. The 2x2 binary 
patterns form the watermarks that are embedded in the halftone output image. The 
halftone output image containing the visible watermark is then output, via 
input/output interface 1 10 and the link 310 to the image data sink 300. 

[0041] It should be appreciated that the image data sink 300 can be any device or 

combination of devices that outputs or displays the resulting halftone image data. 
Thus, the resulting image can be a printed version of the input image, and the image 
data sink 300 can be a printer. Similarly, the image data sink 300 can be a monitor 
which is capable of displaying the electronic version of the resulting halftone image 
for viewing. Furthermore, the image data source 200, the watermarked image 
generator 100, and the image data sink 300 can be elements integrated into a single 
device, such as a digital photocopier. 

Similarly, the image data sink 300 can be a server or other node on a local area 
network, a wide area network, an intranet, the Internet, or any other distributed 
network. In this case, the resulting halftone image is transferred and stored on the 
network in electronic form. Finally, the link 31 0 can be a wired or wireless link to the 
image data sink 300, to any other image output device or display device, or to the 
network. Thus, the image data sink 300 and the link 31 0 can be any known or later- 
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developed systems and methods that are capable of receiving, and outputting or 
storing, the resulting electronic halftone image data generated by the watermarked 
image generator 100. 

[0043] Fig. 2 shows a first exemplary embodiment of the watermark embedding device 
1 50 according to this invention. As shown in Fig. 2, the watermark embedding device 
1 50 includes a tri-level error diffusion generator 1 51 , a decision circuit 1 52, and 
halftoning circuits 1 53, 1 55, 1 57 and 1 59. As will be described in greater detail, the 
watermark embedding device 1 50 halftones the input image to output an image made 
up of 2x2 binary patterns by using a two-stage halftoning process with error 
diffusion. 

[0044] The image input device 200 inputs the original grayscale or continuous tone 

image data to the tri-level error diffusion generator 151. The tri-level error diffusion 
generator 151 halftones the original continuous tone image data by tri-level error 
diffusion into three gray levels. This is performed at a reduced resolution. Specifically, 
each of the 2x2 pixels of the input image are grouped into one super pixel. The value 
of the super pixel is defined to be the average of the 2x2 pixels. The error diffusion is 
then performed on the super pixels. The three gray levels to which the reduced- 
resolution continuous tone image is halftoned are black, white, and 50% gray. The 
process of error-diffusion, including tri-level and other multi-level error diffusion, is 
well known in the art. In general, any known or later-discovered error diffusion 
process is suitable for use with the systems and methods according to this invention. 

[0045] 

After the input image is halftoned by the tri-level error diffusion generator 151, 
the halftoning circuits 1 53, 1 55, 157 and 159 replace each one of the pixels of the 
halftoned input image by a corresponding one of four 2x2 binary patterns. Which of 
the four 2X2 binary patterns is chosen to replace a particular pixel depends on 
whether the particular halftone pixel from the tri-level error diffusion generator is 
black, white, or 50% gray, and/or on the value of the binary watermark stored in or 
input to the watermarked image generator 1 50 for that particular pixel. The controller 
120 sequentially feeds each bit of the binary watermark data stored in the memory 
1 30 or input by the user through the one or more user input devices 1 40 to the 
decision circuit 1 52. The source of the binary watermark data is shown generally in 
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Fig. 2 as a binary data source 400. 

[0046] When a pixel of the haiftoned input image is black, the decision circuit 1 52 is 

bypassed and the halftoning circuit 1 53 replaces the black pixel with an all black 2x2 
binary pattern, as shown in part (a) of Fig. 3. When a pixel of the haiftoned image is 
white, the decision circuit 1 52 is also bypassed and the halftoning circuit 1 59 replaces 
the white pixel with an all white 2x2 binary pattern, as shown in part (b) of Fig. 3. 
When a pixel is 50% gray and the next bit of the binary watermark data from the 
binary data source 400 is 1 , the halftoning circuit 1 55 replaces the 50% gray pixel 
with a 2x2 binary pattern having black dots forming a pattern having a 45 0 polarized 
angle, as shown in part (c) of Fig. 3. When a pixel is 50% gray and the next bit of the 
binary watermark data from the binary data source 400 is 0, the halftoning circuit 1 57 
replaces the 50% gray pixel with a 2x2 binary pattern having black dots forming a 
pattern having a 1 35 0 polarized angle, as shown in part (d) of Fig. 3. 

[0047] It should be appreciated that which 2x2 binary pattern chosen to replace the 
current pixel when the corresponding bit of the binary watermark data have a 
particular value is not limited to that shown in Fig. 4. For example, when a bit of the 
binary watermark data is 0, the corresponding pixel can be replaced with a 2x2 binary 
pattern having a 45 ° polarized angle. 

[0048] The 2x2 binary patterns for the 50% gray level have two different orientations. 

However, they have the same gray appearance, which allows the watermark to remain 
hidden in the final image. The tri-level halftone output from any continuous-tone 
input, except solid black, solid white and 50% gray inputs, will be a spatial 
combination of the intermediate-level bipolar patterns and either the black or the 
white patterns. Toggling the polarization between the two 2x2 binary patterns allows 
binary information to be arbitrarily embedded into any arbitrary image. 

[0049] Fig. 4 is a flowchart outlining a first exemplary embodiment of a method for 
embedding visible watermark into received images according to this invention. 
Beginning in step SI 00, operation continues to step SI 05, where continuous tone 
image data is input. Next, in step SI 1 0, the input continuous tone image data is 
haiftoned by tri-level error diffusion. Operation continues to step S1 1 5. 
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[0050] In step SI 1 5, a pixel is chosen from the halftoned image data. Next, in step SI 20, 
a determination is made whether the pixel of the tri-level output is black. If the pixel 
of the tri-level output is black, operation goes to step SI 25. Otherwise operation 
jumps to step SI 30. In step SI 25, the black pixel is replaced with an all black 2x2 
binary pattern. Operation then jumps to step SI 60. 

[005 1 ] In step SI 30, a determination is made whether the pixel of the tri-level output is 
white. If the pixel of the tri-level output is white, operation goes to step SI 35. 
Otherwise, operation jumps to step SI 40. In step SI 35 the white pixel is replaced with 
an all white 2x2 binary pattern. Operation then again jumps to step SI 60. 

[0052] In step SI 40, a next bit of binary watermark data is input. Then, in step SI 45, a 

determination is made, if the value of the next bit of the binary watermark data is 0. If 
the value of the next bit of the binary watermark data is 0, operation proceeds to step 
SI 50. Otherwise, operation jumps to step SI 55. In step SI 50, the pixel of the tri-level 
output is replaced with a 2x2 binary pattern having a 1 35 ° polarized angle. Operation 
then jumps to step SI 60. In contrast, in step SI 55, the pixel of the tri-level output is 
replaced with a 2x2 binary pattern having a 45 ° polarized angle. Operation then 
continues to step SI 60. 

[0053] In step SI 60, a determination is made whether any pixels are left from the tri- 
level output. If there are pixels remaining, operation returns to step SI 1 5, where 
another pixel is chosen from the tri-level output. If there are no pixels remaining, 
operation continues to step SI 65, where operation of the method ends. 

[0054] Figs. 5A-5D show patches of images after being watermarked by the methods and 
systems according to this invention. In Figs. 5A-5D, the 2x2 binary pattern shown in 
part (c) of Fig. 3 is chosen to replace a 50% gray pixel. 

[0055] Fig. 5A shows the output of the watermarking embedding device according to this 
invention when the constant input represents a dark shadow region, or 0<u<<1 28, 
where u is the gray level of the input based on a 256 level scale. In this case, the white 
dots form patterns having a 45 ° polarized angle. 

[0056] pig. 5D shows the output of the watermarking embedding device according to this 
invention when the constant input represents a highlight region, or 1 28<<u<255. In 
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this case, the black dots form a pattern having a 1 35 ° polarized angle. 

[0057] When the gray level u of the constant input is exactly equal to 1 28, a perfect 

checkerboard pattern will form, and no information can be embedded. However, when 
the gray scale level u of the constant input is close to, but not equal to, 128, 
information is embedded in the input image at a larger scale. For example, Fig. 5D 
shows the output of the watermarking embedding device according to this invention 
when the constant input represents a dark mid-tone region, or 0<<u<1 28. In this 
case, large size black dots are formed having a 45 ° polarized angle. Fig. 5C shows the 
output of the watermarking embedding device according to this invention when the 
constant input represents a light mid-tone region, or 128<u<<255. In this case, large 
size white dots are formed having a 1 35 0 polarized angle. 

[0058] The following illustrative examples are made in reference to the image shown in 
Fig. 6. Fig. 6 shows an error diffused image of a chapel printed at 600 dpi. 

Example 1 : 

[0059] The watermark embedding method according to this invention is coded in C. All 
test images are halftoned and printed at 600 dpi, and then scanned at 600 spots per 
inch (spi). First, the image shown in Fig. 6 is halftoned by the method according to 
this invention using only the 2x2 pattern shown in part (c) of Fig. 3 to replace the 
intermediate level. Fig. 7 A shows a portion of the digital version of the image shown 
in Fig. 6 after being watermarked by the method according to this invention. Fig. 7B 
shows the same portion of the image shown in Fig. 7 A after being printed and 
scanned. The four distinct polarized structures shown in Figs. 5A-D are visible in the 
images of Figs. 7A and 7B. 

Example 2: 

[0060] The same as Example 1 , except only the 2x2 pattern shown in part (d) of Fig. 3 is 
used to replace the intermediate level. Fig. 8A shows a portion of the digital version of 
the image shown in Fig. 6 after being watermarked by the method according to this 
invention. Fig. 8B shows the same portion of the image shown in Fig. 8A after being 
printed and scanned. Figs. 8A and 8B are identical to Figs. 7A and 7B, except for the 
different polarization of the watermarks. 
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Example 3: 

[0061] The watermarking method according to this invention is used to embed a bar code 
in the image shown in Fig. 6. One exemplary pattern for the one-dimensional bar 
code is {1 ,0,1 ,1 ,1 ,0,0,1 ,0,1 }. Each bit of the bar code is encoded using two scanlines, 
and the bar code is repeated throughout the image. Two regions of the watermarked 
image are shown in Figs. 9 and 10. The dot structures having different orientations 
are visible in Figs. 9 and 1 0. Fig. 1 1 shows the entire image shown in Fig. 6 
watermarked with the bar code. A comparison of the image shown in Fig. 1 1 with the 
image shown in Fig. 6 illustrates that the watermarking method according to this 
invention does not cause significant degradation of image quality. 

[0062] Fig. 1 2 shows a second embodiment of a watermark embedding device 500 

according to this invention. As shown in Fig. 1 2, the watermark embedding device 
includes a tri-level error diffusion generator 510, a decision circuit 520, and 
halftoning circuits 530, 540, 550, 560, 570 and 580. As in the first embodiment 
shown in Fig. 2, the tri-level error diffusion generator 510 halftones a reduced 
resolution version of the original input continuous tone image data by tri-level error 
diffusion into three gray levels. The three gray levels to which the original image is 
halftoned are black, white, and 50% gray. 

[0063] After the input image is halftoned by the tri-level error diffusion generator 510, 
the halftoning circuits 530, 540, 550, 560, 570 and 580 replace each one of the 
pixels of the halftoned input image by a corresponding one of six 2x2 binary patterns. 
Which of the six 2x2 binary patterns chosen to replace a particular pixel depends on 
whether the particular halftone pixel is black, white, or 50% gray, and/or on the value 
of the binary watermark stored in or input to the watermarked image generator 500 
for that particular pixel. In this embodiment, the controller 1 20 sequentially feeds two 
bits, rather than a single bit as in the first embodiment, of the binary watermark data 
stored in the memory 1 30 or input by the user through the user input device 1 40 to 
the decision circuit 520. The source of the binary watermark data is shown generally 
in Fig. 1 2 as the binary data source 600. 

[0064] when a pjxe) of the halftoned input image is black, the decision circuit 520 is 

bypassed and the halftoning circuit 530 replaces the black pixel with an all black 2x2 
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binary pattern, as shown in part (a) of Fig. 1 3. When a pixel of the halftoned image is 
white, the decision circuit 520 is again bypassed and the halftoning circuit 580 
replaces the white pixel with an all white 2x2 binary pattern, as shown in part (b) of 
Fig. 13. 

[0065] When a pixel is 50% gray and the bits of the binary watermark data are "00," the 
halftoning circuit 540 replaces the 50% gray pixel with a 2x2 binary pattern having 
black dots forming a pattern having a 0 # polarized angle, as shown in part (c) of Fig. 
1 3. When a pixel is 50% gray and the bits of the binary watermark data are "01 the 
halftoning circuit 550 replaces the 50% gray pixel with a 2x2 binary pattern having 
black dots forming a pattern having a 45 ° polarized angle, as shown in part (d) of Fig. 
13. 

[0066] When a pixel is 50% gray and the bits of the binary watermark data are "1 0," the 
halftoning circuit 560 replaces the 50% gray pixel with a 2x2 binary pattern having 
black dots forming a pattern having a 90 ° polarized angle, as shown in part (e) of Fig. 
1 3. When a pixel is 50% gray and the bits of the binary watermark data are "1 1 the 
halftoning circuit 570 replaces the 50% gray pixel with a 2x2 binary pattern having 
black dots forming a pattern having a 1 35 ° polarized angle, as shown in part (f) of 
Fig. 13. 

[0067] According to the second embodiment, the 2x2 binary patterns for the 50% gray 
level have four different orientations. However, they have the same gray appearance, 
which allows the watermark to remain hidden in the final image. The tri-level halftone 
image data output from any continuous tone input image, except solid black, solid 
white and 50% gray inputs, will be a spatial combination of the intermediate-level 
bipolar patterns and either the black or the white. Toggling the polarization between 
the four 2x2 binary patterns allows binary information to be arbitrarily embedded into 
an arbitrary image. 

[0068] Fig. 1 4 is a flowchart outlining a second exemplary embodiment of a method for 
embedding visible watermark into received images according to this invention. 
Beginning in step S200, operation continues to step S205, where continuous image 
data is input. Next, in step S21 0, the input continuous tone image data is halftoned by 
tri-level error diffusion. In step S21 5, a pixel is chosen from the halftoned image data. 
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Next, in step S220, a determination is made whether the pixel of the tri-level output 
black. If the pixel of the tri-level output is black, operation goes to step S225. 
Otherwise, operation jumps to step S230. In step S225, the black pixel is replaced 
with an all black 2x2 binary pattern. Operation then jumps to step S280. 

[0069] In step S230, a determination is made whether the pixel of the tri-level output is 
white. If the pixel of the tri-level output is white, operation goes to step S235. 
Otherwise, operation jumps to step S240. In step S235, the white pixel is replaced 
with an all white 2X2 binary pattern. Operation then jumps to step S280. 

[0070] In step S240, the next two bits of binary watermark data are input. Then, in step 
S245, a determination is made if the value of the next two bits of the binary 
watermark data are 00. If the value of the next two bits of the binary watermark data 
are 00, operation proceeds to step S250. Otherwise, operation jumps to step S255. In 
step S250, the pixel of the tri-level output is replaced with a 2x2 binary pattern 
having a 0 ° polarized angle. Operation then jumps to step S280. 

[0071] In step S255, a determination is made if the value of the next two bits of the 

binary watermark data are 01 . If the value of the next two bits of the binary watermark 
data are 01 , operation goes to step S260. Otherwise, operation jumps to step S265. In 
step S260, the pixel of the tri-level output is replaced with a 2x2 binary pattern 
having a 45 9 polarized angle. Operation then jumps to step S280. 

[0072] In step S265, a determination is made if the value of the next two bits of the 

binary watermark data are 1 0. If the next two bits of the binary watermark data are 
10, operation goes to step S270. Otherwise, operation jumps to step S275. In step 
S270, the pixel of the tri-level output is replaced with a 2x2 binary pattern having a 
90 ' polarized angle. In contrast, in step S275, the pixel of the tri-level output is 
replaced with a 2x2 binary pattern having a 1 35 ° polarized angle. Operation then 
continues to step S280. 

[0073] In step S280, a determination is made whether any pixels are left from the tri- 
level output. If there are pixels remaining, operation returns to step S21 5, where 
another pixel is chosen from the tri-level output. If there are no pixels remaining, 
operation continues to step S285, where operation of the method ends. 
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[0074] It should be appreciated that which 2x2 binary pattern chosen to replace the 
current pixel when the corresponding two bits of the binary watermark data have 
particular values is not limited to that shown in Fig. 14. For example, when the bits of 
the binary watermark data are 00, the corresponding pixel can be replaced with a 2x2 
binary pattern having a 45 ° polarized angle. 

[0075] The various exemplary embodiments of the systems and methods according to 
this invention can be used to embed many types of information in an original image. 
For example, the methods and systems according to this invention can be used to 
embed a segmentation map in a printed document image. Document images can be 
segmented into regions that have distinct properties, such as text, pictures and 
graphics. A segmentation map is an indexed image, where the index of a pixel is the 
class label of a corresponding pixel or a corresponding group of pixels in the original 
image. Because regions having different characteristics need to be processed and 
rendered differently, image segmentation is crucial for generating or re-producing 
high quality document images. 

[0076] Conventionally, the segmentation map of a document image is often only available 
before the image is printed. However, it is discarded when the image is rendered. If 
the printed image is digitally scanned, the segmentation map needs to be generated 
again, since it is difficult to recover from the printed document. Thus, it is desirable to 
embed the segmentation map in the printed document image for fast and accurate 
retrieval. 

[0077] A segmentation map can be embedded into an image either with encoding or 

without encoding. When a segmentation map is not encoded, the segmentation map is 
directly embedded into an image. In this case, the segmentation map is directly 
correlated with the image content. Thus, even if the printed image is altered, such as 
cropped, or rotated, the segmentation map can still be recovered. Embedding the 
segmentation map without encoding also improves robustness to detection errors. 
That is, in this case, the effect of a wrong detection can be locally contained. 

[0078] on the other hand, by embedding with encoding, the amount of data that needs 
be embedded can be drastically reduced. Fig. 1 5 shows a segmentation map 
embedding device that encodes a segmentation map before embedding the 
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segmentation map into an image. The segmentation map embedding device 600 
includes an image data source 610, a segmentation map generator 620, an encoder 
630, and a watermarked image generator 640. The image data source 61 0 inputs a 
digital image to the segmentation map generator 620. The segmentation map 
generator 620 generates a segmentation map of the image input by the original image 
source 61 0. The segmentation map is then input to the encoder 630. The encoder 
encodes the segmentation map. The encoder 630 can encode the segmentation map 
by any suitable encoding process. For example, if all regions in a document image are 
assumed to be rectangular in shape, then each region can be coded using four 
numbers that specify the location and the dimension of a rectangular, followed by a 
class label. Generally, a segmentation map can be first compressed using either run- 
length coding, or predictive coding, followed by entropy coding. 

[0079] After the segmentation map is encoded by the encoder 630, the encoded 
segmentation map is treated as a binary data source. The binary data from the 
encoded segmentation map is input to the watermarked image generator 640. The 
watermarked image generator 640 embeds the binary data from the encoded 
segmentation map in the image as watermarks using the various exemplary 
embodiments of the systems and methods according to this invention. 

[0080] Fig. 1 6 is a flowchart outlining the embedding of binary segmentation values into 
an input image using the second embodiment of the method for embedding a 
watermark into an image according to this invention. Beginning in step S300, 
operation continues to step S305 where continuous tone image data is input. Next, in 
step S3! 0, the input continuous tone image data is halftoned by tri-level error 
diffusion. In step S3! 5, a pixel is chosen from the halftoned image data. Next in step 
S320, a determination is made whether the pixel of the tri-level output is black. If the 
pixel of the tri-level output is black, operation goes to step S325. Otherwise, 
operation goes to step S330. In step S325, the black pixel is replaced with an all black 
2x2 binary pattern. Operation then jumps to step S380. 

[0081] 

In step S330, a determination is made whether the pixel of the tri-level output is 
white. If the pixel of the tri-level output is white, operation goes to step S335. 
Otherwise, operation goes to step S330. In step S325, the white pixel is replaced with 
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an all white 2x2 binary pattern. 
[0082] In step S340, the next two bits of the binary segmentation map data are input. 

[0083] Operation then continues to step S345. in step S345, a determination is made 

whether the values of the next two bits of the binary segmentation map data are 00. if 
the values of the next two bits of the binary segmentation map data are 00, then 
operation goes to step S21 0. Otherwise, operation goes to step S355. In step S350, 
the current pixel is replaced with a 2x2 binary pattern having a 0 ° polarized angle. 
Operation then jumps to step S380. 

[0084] In step S355, a determination is made whether the values of the next two bits of 
the binary segmentation map data are 01 . If the values of the next two bits of the 
binary segmentation map data are 01 , then operation goes to step S360. Otherwise, 
operation goes to step S365. In step S360, the current pixel is replaced with a 2x2 
binary pattern having a 45 ° polarized angle. Operation then jumps to step S380. 

[0085] In step S365, a determination is made whether the values of the next two bits of 
the binary segmentation map data are 10. If the values of the next two bits of the 
binary segmentation map data are 1 0, then operation goes to step S3 70. Otherwise, 
operation goes to step S375. In step S370, the current pixel is replaced with a 2x2 
binary pattern having a 90 ° polarized angle. In contrast, in step S375, the current 
pixel is replaced with a 2x2 binary pattern having a 1 35 * polarized angle. 

[0086] In step S380, a determination is made whether any pixels are left from the tri- 
level image. If there are pixels remaining, operation returns to step S3! 5, where 
another pixel is chosen from the tri-level output. If there are no pixels remaining, 
operation continues to step S385, where operation of the method ends. 

[0087] It should be appreciated that which 2x2 binary pattern chosen to replace the 

current pixel when the corresponding two bits of the binary segmentation map data 
have particular values is not limited to that shown in Fig. 16. For example, if the next 
two bits of the binary segmentation map data are 00, the current pixel can be 
replaced with a 2x2 binary pattern having a 1 35 ° polarized angle. 

[0088] 

The following illustrative examples are made in reference to the image shown in 
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Fig. 17. 

Example 4: 

[0089] The third embodiment of the watermark embedding method according to this 
invention is coded in C and tested. All test images are halftoned and printed at 600 
dots per inch (dpi), then scanned at 600 dpi. Each 2x2 block of pixels is manually 
classified into three classes: pictorial regions, text and graphics on solid background, 
and text and graphics on halftone background. The resulting segmentation map is 
shown in Fig. 18. In Fig. 18, the black regions indicate pictures, the gray regions 
indicate text on halftone background, and the white regions indicate text on solid 
background. The segmentation map is directly embedded in the original image shown 
in Fig. 1 7, without encoding. The resulting watermarked image is shown in Fig. 19. 
The three segmentation classes are represented by the 45 0 , 90 ° and 1 35 ° 2X2 
polarized patterns shown in parts (d)-(f) of Fig. 13, respectively. 

[0090] A comparison of the watermarked image shown in Fig. 19 with the corresponding 
original image shown in Fig. 1 7 illustrates that the watermarking method according to 
this invention does not cause significant degradation of image quality. 

[0091] The retrieval of watermarks embedded by the various exemplary embodiments of 
the systems and methods according to this invention requires only the detection of 
the polarizations of the dot structures on the watermarked image. Thus, retrieval is 
relatively easy and robust. One exemplary embodiment of a method of retrieving 
watermarks embedded by the various exemplary embodiments of the systems and 
methods according to this invention includes scanning the printed image with a 
scanner having a resolution at least equal to the resolution of the embedded 
watermark. The watermarks in the digitized image can be detected using pattern 
matching methods that are well known in the art. 

[0092] 

Fig. 20 is a flowchart outlining one exemplary embodiment of a method of 
retrieving watermarks embedded by the various exemplary embodiments of the 
systems and methods according to this invention. Beginning in step S400, operation 
continues to step S405, where the watermarked image data is input. Next, in step 
S410, a 2x2 pixel pattern is chosen from the watermarked image data. Next, in step 
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S41 5, a determination is made whether the 2x2 pixel pattern is all white or all black. If 
the 2x2 pixel pattern is all white or all black, operation goes to step 455. Otherwise, 
operation goes to step S420. 

[0093] In step S420, a determination is made whether the 2x2 binary pattern has a 0 ° 

polarized angle. If the 2x2 binary pattern has a 0 ° polarized angle, operation goes to 
step S425. Otherwise, operation goes to step S430. In step S425, the values of the 
next two bits of the watermark data are determined to be 00. Operation then jumps to 
stepS455. 

[0094] In step S430, a determination is made whether the 2x2 binary pattern has a 45 9 

polarized angle. If the 2x2 binary pattern has a 45 * polarized angle, operation goes to 
step S435. Otherwise, operation goes to step S440. In step S435, the values of the 
next two bits of the watermark data are determined to be 01 . Operation then jumps to 
step S455. 

[0095] In step S440, a determination is made whether the 2x2 binary pattern has a 90 ° 

polarized angle. If the 2x2 binary pattern has a 90 ° polarized angle, operation goes to 
step S445. Otherwise, operation goes to step S450. In step S445, the values of the 
next two bits of the watermark data are determined to be 1 0. In contrast, in step 
S450, the values of the next two bits of the watermark data are determined to be 1 1 . 
Operation then continues to step S455. 

[0096] In step S455, a determination is made whether any pixels are left from the 
watermarked image data. If there are pixels remaining, operation returns to step 
S41 0, where another 2x2 binary pattern is chosen from the watermarked image data. 
If there are no pixels remaining, operation continues to step S460, where operation of 
the method ends. 

[0097] It should be appreciated that which values for the next two bits of the watermark 
chosen to correspond to the 2x2 binary pattern of the binary watermarked image data 
is not limited to those shown in Fig. 20. For example, if the 2x2 binary pattern has a 
90 ° polarized angle, the values of the next two bits of the watermark data can be 
determined to be 1 1. 

[0098] whi | e thjs invention has been described in conjunction with the specific exemplary 
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embodiments outlined above, it is evident that many alternatives, modifications and 
variations will be apparent to those skilled in the art. Accordingly, the exemplary 
embodiments of the invention, as set forth above, are intended to be illustrative, not 
limiting. Various changes may be made without departing from the spirit and scope of 
the invention. 
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